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A  sequence  of  intennediate  products  was  developed  over  a  series  of  workshop  sessions  that  lead  to  the  final  workshop  report. 
Session  1 :  Service  visions  of  future  T&E  most  effectively  and  efficiently  utilizing  M&S 

Sessions  2&3:  Integrated  service  visions  and  elaborated  needs 

Session  4:  Policy.  Management  and  Technical  prioritized  sets  of  needs  and  final  briefing 


Session  S:  Service  discussions,  impacts,  and  recommendations 

SUMMARY  OF  WORKSHOP-IDENTinED  T&E  MAS  REQUIREMENTS 
I .  Verification,  Validation,  and  Accreditation  (VVA  A) 
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DMSO  is  to  encotuage  development  of  models  and  simulations  that  inanporate  both  vertical  and  horizontal 
integration. 

Services  are  encouraged  to  use  nrKxlels  and  simulations  that  are  capable  of  simulating  joint  operations. 


1.1 


DMSO  is  to  support  use  of  M&S  in  T&E  by  development  of  a  M&S  developers  handbook. 


■ .  • 


The  Defense  Modeling  and  Simulation  Office  (DMSO)  sponsored  the  Test  and  Evaluation  (T&E)  Modeling  and  Simulation  (M&S) 
Workshop,  reported  in  this  document,  as  a  forum  to  identify  T&E  M&S  requirements. 
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T&E  WORKSHOP  OBJECTIVES 


WORKING  GROUP  PROCESS  AND  PRODUCTS 

—  WORKING  GROUPS  — 
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WORKING  GROUP  PROCESS  AND  PRODUCTS 

—  SESSIONS  — 
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WORKING  GROUP  PROCESS  AND  PRODUCTS 
—  GROUPING  OF  ACTIVITIES  — 
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Private  Industry 

Activities  that  should  be  leveraged  by  the  T&E  community 


WORKING  GROUP  PROCESS  AND  PRODUCTS 
—  GROUPING  OF  ACTIVITIES  — 
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Technical  actions 


WORKING  GROUP  PROCESS  AND  PRODUCTS 
—  TYPES  OF  ACTIVITIES  IDENTIFIED 
IN  SENIOR  EXECUTIVE  PANEL  REVIEW  — 
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SUMMARY  RESULTS: 
VERIFICATION,  VALIDATION,  AND 
ACCREDITATION  (VV&A) 
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SUMMARY  RESULTS; 

VERIFICATION,  VALIDATION,  AND  ACCREDITATION  (VV&A) 


SUMMARY  RESULTS: 

STANDARD  M&S  AND  STANDARD  DATABASES 
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SUMMARY  RESULTS: 

STANDARD  M&S  AND  STANDARD  DATABASES 
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SUMMARY  RESULTS: 
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(M)  Through  the  Defense  Acquisition  University,  educate  the  acquisition  work  force  and  decision  makers  on  the 
utility  of  MAS  for  T&E 


SUMMARY  RESULTS: 
INFORMATION  SHARING 
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SUMMARY  RESULTS: 
NETWORKS  AND  JOINTNESS 
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SUMMARY  RESULTS: 
NETWORKS  AND  JOINTNESS 
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(P)  EnowOTge  dcyelopnynt  of  models  and  simulations  that  are  capable  of  simulating  joint-service  operations  or  ths 
are  uiteroperaUe  with  such  simulations  in  order  to  assess  the  contributions  of  weapon  systems  within  the  Joint 
operations  environments 


SUMMARY  RESULTS: 
MULTI-LEVEL  SECURITY 
AND  PLANNING  FOR  THE  USE  OF  M&S 
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SUMMARY  RESULTS: 

MULTI-LEVEL  SECURITY  AND  PLANNING  FOR  THE  USE  OF  M&S 


SUMMARY  RESULTS: 
M&S  DEVELOPMENT 
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SUMMARY  RESULTS: 
INDUSTRY  DEVELOPMENTS 
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SUMMARY  RESULTS: 
INDUSTRY  DEVELOPMENTS 
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SUMMARY  RESULTS: 
DEVELOPERS  HANDBOOK 
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SUMMARY  RESULTS: 
TOOLS  &  HANDBOOK 
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Some  activities  should  be  performed  by  and/or  funded  by  one  of  tte  defense  agencies  for  use  external  to 
the  agency  -  T&E  community  should  take  advantage  of  (leverage)  these  activities 
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APPENDIX  C 
VISION 


ARMY  WORKING  GROUP 


ARMY  WORnNQ  GROUP 


VISION 


ARMY  WORKING  GROUP 

OPPORTUNITIES/NEEDS 


"The  Army  wants  to  be  able  to ..." 

•  Test  &  evaluate  non-traditional  threats  (e.g., 
SO/UC)  and  associated  spectra  of  conflict 

•  Share  T&E  models  and  data  among  the  DoD 
activities  and  among  the  Government 

agencies  (e.g.,  NASA,  Commerce,  ITS) 

•  Use  MAS  to  increase  support  of  T&E 
planning,  conduct,  data  reduction,  real-time 
assessment,  analysis,  reporting  and 
distribution 
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ARMY  WORKMQ  GROUP 

OPPORTUNITIES/NEEDS  (Concluded) 


•  Create  a  Joint  M&S  capability  arhich  directly 
reiatea  teat  results  to  combat  effectiveness 
for  all  weapon  systems  (e.g.,  C3i,  Intel 
Fusion,  Human  Factors) 

•  Build  life-cycle  simulations,  early  on,  in 
support  of  T&E  which  also  support  all  other 
phases  of  acquisition,  training,  and 
deployment 

•  Create  M&S  to  support  T&E  of  large-scale 
composite  systems  (e.g.,  GPALS) 


ARMY  WORKING  GROUP 

CAPABILITIES 


"The  capabilities  required  by  M&S  to  meet  the 
opportunities  are:” 

•  Management  with  a  vision  of  the  value  of 
M&S  in  T&E 

•  CaoMbllltv  to  perform  Real-Time  Data 
Collection.  Reduction.  Vlauallzatlon.  and 
Distribution  from  simulation,  testing,  and 
live  exercises 

‘  ^standardized  databases,  support  modela 
and  software  methodolooles  to  Improve  the 
efficiency  and  effectiveness  of  M&S  In 
support  of  T&E 

C-4 


AfOlY  WORKINQ  GROUP 

CAPABILITIES  (ConcludGd) 


•  Notwork  capabilitiM  to  tranafar  &  procass 
larga  volumaa  of  data  from  nattad  aystams 
tasting  (a.g.,  C3I,  Space) 
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Training  In  all  disdplinas  related  to  M&S 


ARMY  WORKING  GROUP 

INFRASTRUCTURE  VISION 


The  characteristics  of  an  infrastructure  to 
enhance  integration  of  the  M&S  Community 
in  the  T&E  Community  are:” 

Communications/Distributed  Data  Systems 
/  Desktop  Systems  to  share  information 
(e.g.,  status,  reviews)  •  FREE 

•  Training  (e.g.,  via  model  user  groups) 

Centers  of  Exceiience/Service  Proponency 
Office 


ARyy  WORKINQ  GROUP 

INFRASTRUCTURE  VISION  (Conducted) 


•  Incentives  for  sharing 

•  Mechanism  for  eaily  Involvement 

•  Streamline  ADPE  procurement  process 


ARMY  WORKINQ  GROUP 

ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION 


Characteristics  to  support  the 
interoperability  of  M&S  across  the  T&E 
Community  are:” 

*  Open  Systems  Architecture 

*  JMASS*like  Architecture 

*  Standardized/Coordinated  Model  Design 
(e.g.,  CASE  Tools) 

*  Standards  must  be  responsive  to  user 
needs  (e.g.,  ADA) 


C4 


ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION  (Concluded) 


'Horizontal'  and  'Verticar  interoperability  / 
interfaces 


Standard  nomenclature  for  objects 
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APPENDIX  D 
VISION 


NAVY  WORKING  GROUP 


NAVY  WORKMa  GROUP 


VISION 


NAVY  WORKMG  GROUP 

FUTURE  OPPORTUNITIES 

•  Reduce  Cost  Of  T&E 

•  Consolidate  resources 

•  Affordable  Interoperability 

•  Testing  the  Untestable 

•  Expand  testing  envelope 

•  Live  fire  testing 

•  Earlier  Involvement  of  Testers  &  Users 

•  Earlier  Op  assessments 

•  Evaluate  design  early 


NAVY  WORKMQ  GROUP 

FUTURE  OPPORTUNITIES  (ContiniMd) 


-  Eariy  dev  of  deployment  tactics 

-  Support  source  selection  process 

*  Increased  System  Characterization  Robustness 

•  Augment  real  test  data 

-  Expand  testing  envelope 

•  Component  eval  in  broader  warfighting 
context 

-  Sim  as  rigorous  extension  of  eng  dev 

*  improved  Test  Design 

•  Conserve  test  assets 


NAVY  WORKSIO  GROUP 

FUTURE  OPPORTUNITIES  (Continued) 

•  Improved  System  Design 
•  SW  IV&V 

•  Increased  Efficiency  of  Life-Cycle 
Additional  Ungrouped  Opportunities 

•  Replay  &  post-op  trouble  shooting 

•  Characterize  systems  through  modeiing  and 
validate  models  through  testing 


Overcome  T&E,  site  limitations 


NAVY  WOfOONQ  GROUP 

FUTURE  OPPORTUNITIES  (Conducted) 


Additional  Ungrouped  Opportuntties 

•  Reduce  duplication  of  T&E  capabilities 

•  Improve  inter/intra  service  co^p 

•  Audit  trail  from  ctesign-eval  tools 

•  Seamless  DT-OT  transition 

•  Fix  M&S  dev  responsibilities 


NAVY  WORKING  GROUP 

_ CAPABILITIES  VISION _ 

•  Cheaper  and  More  Capable  HW 

•  New/High  Fidelity  Modeling  Capabilities 

•  High  environmental  fidelity 

•  Varying  fidelity 

•  Non-traditional 

-  Multi-spectral 

•  Near  Field  Target  Signature  Effects 

•  Real-Time  Man/HW  in  the  Loop  Case 

•  CASE 

-  CM  &  doc  Control 
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NAVY  WORKMQ  GROUP 

CAPABILITIES  VISION  (Concluded) 

Model  Interoperability  Techniques 

•  Ease  of  training  &  use 

•  Tie-In  to  related/support  systems 

•  Universal  bridge 

•  Intel  compatibility  checker 

Multi-level  security  (high  data  rate) 
Intelligent  data  bases 
Optimal  test  planning 


NAVY  WORKING  GROUP 

INFRASTRUCTURE  VISION 

Single  Point  of  Contact  for  Each  Service 
Standardized  Cost  Estimating/Accounting 
Sim  User  Organization 
Standardized  Operational  Procedures 
Readiness  Checking/Reporting 
Index  of  Models  w/  Oescriptions 
Central  Conflict  Resolution 


NAVY  WORKMQ  GROUP 

INFRASTRUCTURE  VISION  (Concluded) 


•  Easily  Accessible  from  Model  Catalog 

•  Rapid,  Responsible  Accreditation  Process 

•  Interference  Effects  of  Models 

•  Management  &  Control  of  Models 

•  Sponsor  Org/Funding  Program  Element 

•  Test  Data  Repository 

•  User  Defined  Test  Data  Packages 

•  Brief  &  Debrief  Capabilities 

NAVY  WORKSIO  GROUP 

ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION 

•  Data  Rates 

•  Shared  Data  Base 

•  Data  Latency 

•  Encryption 

•  Selectable  Player  View 

•  Sub-Surface  Connectivity 

•  Bandwidth/Compatibility 

O-l* 
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NAVY  WORKMO  GROUP 

ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION 

Data  Links 
Protocols 

Manage/Schedule/Control  Accass/Audit 
Portable/Independant  Nodas 
Common/Flaxibla  Scanarios 
Dynamic  Command  Structure 
On-Llna  Halp 

Opan  Distributed  Arch/Common  Op  Env 
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VISION 


AIR  FORCE  WORKING  GROUP 


AIR  FORCE  WORKMQ  GROUP 


VISION 


c-i 


AIR  FORCE  WORKINQ  GROUP 

VISION 


"A  M&S  capability  so  complete  and 
comprehensive  that  when  a  live  test  Is 
performed,  It  Is  merely  a  demonstration  of 
the  system's  anticipated  performance" 


AIR  FORCE  WORKMQ  GROUP 
OPPORTUNITIES 


•  Mission  Lsvel  Analysis 

•  Requirement  Definition 

•  Test  Planning 

•  Test  Execution 

C-3 

AIR  FORCE  WORKMQ  GROUP 

FUTURE  OPPORTUNITIES 

•  Enhance  test  scenarios 

•  Supplement  field  testing 

•  Improve  test  effectiveness  &  efficiency 

•  Reduce  cost  schedule  &  risk 

•  Enhance  systems  performance  (Identify 
deficiencies) 


AIR  FORCE  WORKMQ  GROUP 

FUTURE  OPPORTUNITIES  (Continued) 


Perform  early  Integration  testing 

Evaluate  high-level  systems  interfaces 

Predict/extrapolate  test  results 

Understand  test  anomalies 

Establish  analysis  hierarchy 

Simulate  when  cant  test  (space/security/ 
safety) 

E4 


AIR  FORCE  WORKING  GROUP 

FUTURE  OPPORTUNITIES  (Concluded) 


Establish  logistics  supportability 
Utilize  virtual  prototyping 


Link  all  test  resources 


AIR  FORCE  WORKMQ  GROUP 

CAPABILITIES  VISION 


MAS  will  need  to  provide: 

*  Portability 

*  Interoperability 

*  Credibiiity 

*  Reaiistic 

*  Distributed  test  capabiiity 

*  Very  high  speed  iii^l^ing 


AIR  FORCE  WORKINQ  GROUP 

CAPABiLITiES  VISION  (Concluded) 


*  Common  architecture 

*  ReaMime  operation 

*  Variable  resolution 

*  Ease  of  use 


C4 


Scaleable 


AIR  FORCE  WORXMQ  GROUP 

INFRASTRUCTURE  VISION 


Characteristics 

*  Cantral  AP  focal  point 

*  Dooantrallzad  axocutlon 

*  Training 

*  DIaelplinad  tost  prooaaa 

*  Modal  raposhory/claarlng  facility 

*  High  parformanca  communications  A  computing 

*  Funding 


AIR  FORCE  WORKMO  GROUP 

INFRASTRUCTURE  VISION  (Concluded) 


Categories  of  Shared  Information 

•  Architecture 

•  Modeis 
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•  Databases 


AIR  FORCE  WORKMQ  GROUP 

ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION 

*  Information  content 

•  Joint  data  dictionary 

*  Inf  ormation  format 

*  Physical  connections 

•  High  speed  network 
-  Minimum  latency 

•  DIS  interface 
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AIR  FORCE  WORKING  GROUP 

ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION  (Concluded) 

•  Open  systems 

•  Real-time/non  real-time  compatibility 


c-u 


APPENDIX  F 


VISION 

DEFENSE  AGENCIES 
WORKING  GROUP 


DEFENSE  WORKINQ  GROUP 


VISION 


DEFENSE  WORKINQ  GROUP 
FUTURE  OPPORTUNITIES 

Incorporate  Technology 
M&S  Framework 

Supplement  T&E/Pre-Test  Design 
Reduce  Duplication 
Common  Data  Sources 
Traceabiiity/Criticaiity  Analysis 


DEFENSE  WORMNOOROUF 


FUTURE  OPPORTUNITIES  (Concluded) 


Modeling  &  Simulation  of  Requirements 

•  Proof  of  Concept 

•  Test  Design  (DT&E,  OT&E) 

•  identify  T&E  Deficiencies 

•  Lower  Costs 

•  Better  Efficiency 


DEFENSE  WORKING  GROUP 

CAPABILITIES  VISION 

•  Databases 

•  Data  dictionary 

•  Data  definitions 

•  Credible  Models  -  Interoperable 

•  Internet  -  Accessibliity/Sources 

•  Interaction  between  providers  and  users  or 
models 

•  Real-Time  Processing  to  Permit  Man-in-the- 
Loop 


DEFENSE  WORKMQQROUP 

CAPABILITIES  VISION  (ContInuMi) 

•  Rapid  Configuration/Varying  Resolution 

•  Ol^ect  Oriented  Code -Reusable 

•  Visual  Programming/Open  Systems 


DEFENSE  WOAKING  GROUP 

CAPABILITIES  VISION  (Concluded) 

•  Visualization 

•  graphic  depictions  (red  carpet) 

-  holographic  displays 

•  Scalable  Results 

•  Scaled  Prototypes 

•  Compute  power  to  permit  M&S  without 
aggregation 
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DEFENSE  WORKMQQIIOUF 
INFRASTRUCTURE  VISION 


*  Software  Research  Fadlltles/Repositorles 

*  Big  Integrated  Net 

*  DIstrltNJted  Databases 

*  Multi-Security  Levels 

*  Organization  for  Sttindards  Dissemination 


DEFENSE  WORKING  GROUP 

INFRASTRUCTURE  VISION  (Concluded) 


•  Training 

•  Properly  trained  people 

•  Proper  tool 

-  Grade  structures/skiii  requirements 

•  Career  path 


ft 


DEFENSE  WOmCINO  GROUP 

ARCHITECTURE,  STANDARDS 
AND  PROTOCOL  VISION 


•  Classes  of  Architecture  (terrain,  weather, 
system,  doctrine,  t- 1*-^,  N*  *N...) 

*  Standard  Data  Definitions 

*  Standardized  Net  Interfaces 

•  Alternate  Technology  for  Data 
Transmission 


Greater  Compute  Power 


APPENDIX  G 


INTEGRATED  VISION 
AND  NEEDS 

POLICY  WORKING  GROUP 


POLICY  WORKINO  GROUP 


INTEGRATED  VISION  AND  NEEDS 


POLICY  WORKING  GROUP 

GENERAL  OVERVIEW 


Define  Policy  Required  for  DoD  Vision  for  M/S 
Supporting  T&E 

•  Opportunities 


•  Capebilities 


•  Architecture 


•  Infrastructure 
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POLICY  WORKMQ  GROUP 
INTEGRATED  OPPORTUNITIES  VISION 


•  Share  models  and  data  across  DoD 

•  M&S  should  be  used  to  support  the  T&E 
process  throughout  the  system  life  cycle 

•  V&V  design  requirements 

-  development  to  deployment 

-  operational  assessments 

•  M&S  shall  relate  test  results  to  combat 
effectiveness 


C-l 


POUCY  WORKING  GROUP 

INTEGRATED  OPPORTUNITIES  VISION  (Conc'd) 


*  Develop  a  T&E  Master  Plan  to  use  M&S 
through  all  system  milestones 

*  M&S  should  be  used  when  live  tests 
cannot  be  conducted 

*  Establish  central  M&S  authority  to  which 
all  T&E  M&S  activities  must  report 

*  W&A  Policy  for  M&S  (credibility, 
suitability...) 
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POLICY  WORKMQ  GROUP 

INTEGRATED  CAPABILITIES  VISION 


M&S  will  te  doslamdAnodlfiGd  with  the 
following  In  mind:  portability,  Interoper- 
abllTty,  cUltabilltyjiisablllty,  flexibility, 
re^l^llity.  Internetting  and  aecurlty,  and 


M&S  will  consist  of  a  standardized 
documented  version  which  vitll  remain 
Intact;  changes  to  the  standard  will  occur 
to  other  versions 


Encourage  distributed,  reaUtime 
processing  with  Man-in*the-loop/HW  in  the 
loop  capaDility 

Capable  of  playing  roles  in  joint-service 
scenarios  providing  essential  degree  of 
realism 


c-s 


POLICY  WORKING  GROUP 

INTEGRATED  CAPABILITIES  VISION  (Conc’d) 


*  Establish  high  volume  networks  for 
transfer  of  multi-level  classification  of  test 
data 

*  Perform  real-time  data  collection, 
reduction,  visualization,  and  distribution 

*  Establish  "library**  of  validated  M&S 
modules  (at  DoD  or  Service  levels) 

*  Establish  **Lessons-Leamed"  Library 
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POLICY  WORKMQ  GROUP 
INTEGRATED  ARCHITECTURE  VISION 


•  M&S  should  be  devsioped  using  a  common 
opsn  systems  architecture  using  associated 
standards  and  protocols 

•  M&S  Software  should  meet  established 
standards  and  metrics  (i.e.,  MIL*STD*2167A, 
Army  step  process,...) 

•  Require  the  use  of  common  databases  (i.e., 
Intel,  terrain,  atmosphere,  climate,...) 

•  All  M&S  architectures  should  have  the 
capability  to  import  and  export  standard 
databases  produced  by  DoD  OPRs 


POUCY  WORKING  GROUP 

INTEGRATED  ARCHITECTURE  VISION  (Conc'd) 


•  The  contractor  should  provide  non-proprietor 
deliverable  software  package 

•  Producers  of  common  databases  will  comply 
with  standard  database  elements  definitions 

•  M&S  will  allow  vertical  and  horizontal 
integration  (i.e.,  ALSP) 

•  Architecture,  standards,  and  protocol  must 
provide  for  variable  fidelity  (as  dictated  by 
user  needs) 
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POLICY  WORKING  GROUP 

ARCHITECTURE  NEEDS 

•  Standardized  data  definitions 

•  Standard  database  elements 

•  internet  should  be  capable  of  providing 
data  links  for  sub^surface/surface/air 


.  POUCY  WORKING  GROUP 

'  INTEGRATED  INFRASTRUCTURE  VISION 


•  All  M&S  used  in  support  of  T&E  will  be 
accredited  by  the  user 

•  doD  components  shall  establish  V,  V&A 
policy,  procedures,  and  responsibilities 

•  M&S  users/developers  shall  use  valid  data 

•  M&S  developers  will  coordinate  data 
requirements  with  DoD  OPR  prior  to 
expenditure  of  funds 

•  Training:  A  M&S  career  track  will  be 
established  to  Include  M&S  course  work 


C-lt 


APPENDIX  H 


INTEGRATED  VISION 
AND  NEEDS 

MANAGEMENT  WORKING  GROUP 


MANAGEMENT  WORKINQ  GROUP 


INTEGRATED  VISION  AND  NEEDS 


MANAGEMENT  WORKING  GROUP 

From  a  Managament  Parspactive,  Modaling 
&  Simulation  in  Support  of  Tast  &  Evaluation  Should: 

•  Raduca  Costs 

•  Raduca  Risks 

•  incraasa  Efficiancy 

•  Raduca  Schaduias 


improva  T&E  Procass  Credibility 


To  Attain  Thesa  Goals,  tha  Following  Must  Ba  Dona: 

•  Incraasa  Usa  of  M&S  In  T&E 

•  Shara  Modals  &  Data 

•  Standardiza  W&A  Procass 

•  Provida  Infrastructuras  for  Information  & 
Intaroparability 

MANAGEMENT  WORKING  GROUP 

Increase  Use  of  M&S  in  T&E 

•  Promota  Early  Involvamant 

•  Provida  Cantrallzad  Coordination 
and  Allow  for  Dacantralizad  Exacution 

•  Establish  Sarvica  Proponancy 

•  Expand  to  all  Phasas  of  T&E 

•  Provida  Rasourcas 


MANAGEMENT  WORKINQ  GROUP 


Share  Models  and  Data 


•  Standardize: 

•  architecture 
-databases 

•  definitions 

•  documentation 

•  Educate 

-  pubiicize 

•  train 

•  Distribute 

•  Resource 
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MANAGEMENT  WORKING  GROUP 

Provide  Infrastructure  for 
Information  &  Interoperability 

Deveiop  &  Maintain  Modeis/Data  Bases 


improve  Network 


Estabiish  Muiti-ievei  Security  Capabiiity 


Deveiop  &  Use  Muiti-service  Expertise 


MANAGEMENT  WORKING  GROUP 


Standardize  W&A  Process 


H-T 


APPENDIX  I 

INTEGRATED  VISION 
AND  NEEDS 


TECHNICAL  WORKING  GROUP  1 


TECHNICAL  WORKINQ  GROUP  #1 


INTEGRATED  VISION  AND  NEEDS 


M 


TECWaCAL  WORKING  GROUP  *1 


VISION 


Us0  M&S  technology  for  affordable,  timely, 
and  cost  effective  T&E  as  part  of  the  total 
weapon  system  life  cycle  through  reusable 
components  Interoperating  in  an  open 
system  architecture. 
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TECHMCAL  WOfUONQ  GROUP  f1 

INTEGRATED  OPPORTUNITIES  VISION 


•  Um  simulations  to  do  things  that  can't  be 
dona  with  field  tests 

•  site  limitations 

•  safety/security  Issues 

•  large  scale  composite  systems 

•  environmental  concerns 

•  expand  test  envelope 

•  live  fire  testing 

•  interoperability 


TECHNICAL  WORKING  GROUP  «1 
INTEGRATED  OPPORTUNITIES  VISION 


*  Share  service  assets 

-  common  data  sources 

-  Joint  scenarios 

-  reduce  duplication 

-  models  &  data 
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TECHMCAL  WORKING  GROUP  «1 
INTEGRATED  OPPORTUNITIES  VISION 


*  Improve  acquisition  test  pianning 

-  requirements  (systems  &  test)  definition 

•  identify  T&E  deficiencies 

•  traceabiiity/criticality  anaiysis 

-  audit  traii  from  design-evai  tools 

•  test  effectiveness  &  efficiency 


TECHNICAL  WORKING  GROUP  »1 

INTEGRATED  OPPORTUNITIES  VISION 


*  Increase  systems  characterization 
robustness 

•  augment  test  results 
-  conserve  test  assets 

•  eval  in  broader  warfighting  context 

(including  logistics) 

•  scaieability/scope 


TECHNICAL  WORKINQ  GROUP  #1 
INTEGRATED  OPPORTUNITIES  VISION 


*  Enable  virtual  prototyping 

-  proof  of  concept 

•  technology  incorporation 

-  aupport  early  integration  testing 

-  early  and  continuous  involvement  in  life 

cycle 

-  early  involvement  of  tactics  development 

•  support  source  selection  process 

•  transition  to  training 

•  SW IV  &  V 

•  seamless  DT  &  OT 

•  high-level  systems  interfaces 


TECHNICAL  WORKINQ  GROUP  «1 

INTEGRATED  OPPORTUNITIES  VISION 


•  Support  test  execution 

•  data  reduction,  analysis 

-  reporting  and  distribution 

•  understand  test  anomalies 

•  repiay/post-op  trouble  shooting 

-  link  test  resources 


TECHMCAL  W0RKIN6  GROUP  #1 

INTEGRATED  CAPABILITIES  VISION 


*  Databases 

•  standardisation 

•  common  data  on*line  dictionary 

•  inteliigent 

•  common  threat,  environment,  terrain,  etc. 

•  validation 

-  CM,  maintenance  and  distribution 

•  multMevel  security 
.  accessibility 

.  support  multi-level  fidelity 


TECHNICAL  WORKING  GROUP  «1 

INTEGRATED  CAPABILITIES  VISION 


*  Network 

•  very  high  speed 

•  wideband 

•  multMevel  security 

•  real-time 

-  accessibility 


TECHNICAL  WORKING  GROUP  «1 
INTEGRATED  CAPABILITIES  VISION 


•  Models 

-  varying  fidelity 

•  scale/scope 

-  very  high  fidelity 

-  Interoperability  (between  models) 

-  robust  capability 

-  multi-spectral 

-  soft  kill/hard  kill 

•  ease  of  use 

-  portable 


TECHNICAL  WORKING  GROUP  «1 

INTEGRATED  CAPABILITIES  VISION 


Models  (continued) 

•  object-oriented  and  open  systems 
architecture 

-  real-time  operations 

-  man-in-the-loop 

-  hardware-in-the-loop 
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TECHNICAL  WORKING  GROUP  i1 

INTEGRATED  CAPABIUTIES  VISION 


•  Simulation  technology 

-  CASE  tools 

-  visual  programming 

-  open  systems 

-  iow-cost/distrit)Utad/massivaiy>paraliel 
systems 

-  visualisation/imaging  virtual  reality 

•  multi-spectral 

-  methodology  development 

•  intelligent  compatibility  checker 

-  hypermedia 

•  data  compression 


TECHNICAL  WORKING  GROUP  «1 
INTEGRATED  CAPABILITIES  VISION 


•  W&A 

•  Standardized  documentation 

•  formal  CM 

•  on  all  M&S  components 

•  central  service  focal  point 
-  automated  W&A 


TECHMCAL  WORKINQ  GROUP  #1 
INTEGRATED  CAPABIUTIES  VISION 


Expertise 

•  decision  makers  with  realistic  expectations 

-  training  in  all  M&S  disciplines 

-  muiti<disciplined  teams 


TECHNICAL  WORKING  GROUP  i1 
INTEGRATED  ARCHITECTURE  VISION 


Characteristics  of 
architecture  to  support 
interoperabiiity  across  T&E 
community  and  be 
responsive  to  user  needs 
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TECHMCAL  WOmONQ  GROUP  «1 

INTEGRATED  ARCHITECTURE  VISION 


•  Opon  systems 

-  sttunsd  to  Industry  etiangss 

-  ISO 

•  date  Interfscs  stendards 

-  date  dannldon 

-  format 

-  on-lina  tast  casts 

-  ntterork  and  stand-alone 

-  joint  data/obfect  dictionary 


TECHMCAL  WORKMO  GROUP  #1 

INTEGRATED  ARCHITECTURE  VISION 


Supports  standard  SW  development 
methodology  with  SW  support  environment 
Including  automated  documentation 

Compatible  clattet  of  architecture  (terrain, 
weather,  system,  etc) 

Horlsontel  and  vertical  Interoperability/ 
Interfaces 

Supports  distributed  model  design  across 
networks  or  massively-parallel  systems 


TECHMCAL  WORIONQ  GROUP  #1 
INTEGRATED  ARCHITECTURE  VISION 

Physical  connsetlont 

-  DlSIntartecs 

-  tunabls  data  rates 

-  mlnimuin  latency 

Embedded  multHevel  security 
On-line  context-sensitive  help 
On-line  knowledge-based  asset  manager 

M* 


TECHMCAL  WORKING  GROW  #1 

INTEGRATED  ARCHITECTURE  VISION 

Graphical  user  Intertece 
Service  unique  interfaces 

Dynamic  user-defined  environment 

-  player  view 

-  ol^ect  configuration 

-  output  display 

-  test  control 

-  briefing/debriefing  capability 


TICHMCAL  WOmONQ  GROUP  #1 

INTEGRATED  INFRASTRUCTURE  VISION 


•  Establish  modol  A  data  management 

-  repository/clearing  house  (free  and  easy 
access) 

-  standards  development  &  dissemination 

-  simulation  user  organization 

-  formal  CM 

•.  Service  proponency  office 

•  centralized  management 

-  decentralized  execution 


TECHMCAL  WORKING  GROUP  #1 

INTEGRATED  INFRASTRUCTURE  VISION 


Centers  of  excellence 

•  technical  focal  point  for  model  CM 

-  research  into  advanced  and  efficient 
algorithm  development ...  simulation 
technology 

Formal  W&A  process 

-includes  IV  A  V 

-  service  accreditation  agent 
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TECHMCAL  WORKMO  QROUP  #1 

INTEGRATED  INFRASTRUCTURE  VISION 


•  Network 

-  net  management 

-  high  performance 

-  asaet  readineas  check 

-  atandardized  cost  estimated 

•  Training 

•  properly  trained  people 

•  provide  proper  tools 

-  grade  structures/sklil  requirements 

•  career  path 


TECHMCAL  WORKING  GROUP  «1 

INTEGRATED  INFRASTRUCTURE  VISION 


*  Funding/procurement 

•  sponsoring  organization 

-  funding  program  element  (6.2, 6.3,...) 

-  streamlined  ADPE  procurement 

•  Disciplined  scientific  test  process 

-  early  involvement  of  warfighter  and 
T&E  conununity  with  test  process 


APPENDIX  J 

INTEGRATED  VISION 
AND  NEEDS 


TECHNICAL  WORKING  GROUP  2 


TECHMCAL  WOnONQ  GROUP  #2 


INTEGRATED  VISION  AND  NEEDS 
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TECHNICAL  WORKING  GROUP  «2 
INTEGRATED  OPPORTUNITIES  VISION 


•  Share  T&E  models  and  data  among  the  DoD 
activities  and  among  the  Government 
agencies  (e.g.,  NASA,  Commerce,  ITS)  to 
minimize  du^icatlon  of  effort,  reduce  costs, 
and  promote  inter-agency  cooperation 

•  Promote  Interoperability  through  common 
architecture,  standards,  communications 
protocols,  physical  linking  of  T&E  facilities 
and  common/shared  databases 


TECHMCAL  WORKMQ  QROUP  #2 

INTEGRATED  OPPORTUNITIES 
VISION  (Continued) 


•  Um  models  and  simulations  to  M  and 
svaluata 

-  Non  traditional  throats  and  missions 

-  Untastabla(safaty,aacurity,cost, 
snvironmsnt) 

•  Systams  of  systams 

-  Multi-forca/multi-mission 


TECHNICAL  WORKING  QROUP  «2 
OVERALL  VISION  STATEMENT 


*'A  M&S  capability  so  complete  and 
comprahansiva  that  whan  a  live  test  is 
performed,  it  is  merely  a  demonstration  of 
the  system's  anticipated  performance" 


TECHMCAL  WOmONQ  GROUP  #2 

INTEGRATED  OPPORTUNITIES 
VISION  fConcIwted) 


•  DtvGlop  simulator/timulatlon  roprosGRtlng 
thG  systom  In  parallGl  with  tystom 
dGVGlopment  and  maxlmiza  rausa  of 
almulation  modal/modulaa 

•  UaaM&S  throughout  thaayatamlifa-cycia 
to  ineraaaa  aupport  of  T&E  planning, 
conduct,  data  reduction,  raaMime 
aaaaaament,  analysis,  reporting, 
distribution  and  training 

•  M&S  development  In  the  future  will  become 
more  efficient  and  creditable 

J-S 


TECHNICAL  WORKINQ  GROUP  «2 

INTEGRATED  CAPABILITIES  VISION 


•  Models  and  Simulations  shall  operate  in 
multi-level  secure  environments 

•  Modeling  and  Simulation  shall  be 
supported  by  common/shared/intelligent 
databases 
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TECHMCAL  WOfOONO  GROUP  #2 
INTEGRATED  CAPABILITIES 
VISION  rConttnuedl 


•  Models  and  Simulation  architectures 
shall  support  standards  for 
Interoper^mty  and  portability 

•  Models  and  Simulation  hardware  will 
provide  the  power/speed/memory  to 
support  reaMIme  execution, 
visualization,  distribution  from 
simulation,  testing  and  live  fire 


TECHNICAL  WORKING  GROUP  #2 


INTEGRATED  CAPABILITIES 
VISION  (Concluded) 


«  Modeling  and  Simulation  networks  will 
be  capable  of  high-speed  transfer  of 
large  volumes  of  data  from  intemetted 
systems 

•  Models  and  Simulations  shall  be 
scaleable,  variable  resolution,  credible, 
allow  real-time  execution 


TECHMCAL  WORKMQ  GROUP  #2 

CAPABILITIES  NEEDS 


*  M&S  community  iiMds  to  access  on-going 
efforts  to  dsvsiop  mulU-lsvsl  sscurity 
capability  to  aacartain  If  thasa  efforts  will 
meat  specific  needs  of  the  M&S  community 

-  Network 

-  Operating  systems 

-  Databases 

-  DBMS 

•  High  encryption/decryption 
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TECHNICAL  WORKING  GROUP  #2 

CAPABILITIES  NEEDS  (Continued) 


Need  to  develop  capability  to  interface 
different  DBMS 

-  Object  Oriented  vs  RDBMS  (etc) 

Need  a  system  that  will  translate  between 
different  definitions  and  formats 

Need  tools  and  methodologies  to  rapidly 
W&A  databases 

Develop  capability  to  selectively 
access/lnterface/distiibute  data  from 
different  systems/models  including  full  text 


TECHMCAL  WOmONQ  QROUP  #2 

CAPABILITIES  NEEDS  (Continutd) 


Establish  Intsropsrsblilty  standards 


Establish  portability  standards 

Publlsh/train/implamant  standards  in  MAS 
architactura 

Continua  to  coordinata  and  managa  tha 
Idantiflcation  and  Invastlgation  of  now 
tachnology  for  MAS  and  communicata  to 
community  (Usa  focal  points) 


TECHMCAL  WORtONO  GROUP  «2 
CAPABILITIES  NEEDS  (Continued) 


Need  to  provide  transparent,  higher  spaed, 
higher  capacity  network  capM>ia  of 
transmitting  data  in  real  time  to  meat 
special  TAE  needs 

Establish  portability  standards 

PubllsIVtraln/implement  standards  in  MAS 
architecture 

Continue  to  coordinate  and  manage  the 
Identification  and  investigation  of  new 
technology  for  MAS  and  communicate  to 
community  (Use  focal  poims) 


TECHMCAL  WOmONQ  QROUP  #2 
CAPABILITIES  NEEDS  (Concluded) 


Need  to  develop  e  W  A  A  process  for  a 
future  vision  that  stresses  both  stand  alone 
and  Interoperable  distributed  simulations. 

TAE  MAS  need  to  be  developed  or  modified 
to  evaluate  systems  in  their  intended 
operational  environments  to  Include 
non*traditional  systems  (e.g.,  space 
systems)  and  missions 


TECHNICAL  WORKING  GROUP  #2 

INTEGRATED  ARCHITECTURE  VISION 


*  MAS  developed  using  sound  software 
engineering  principles  for  open  system 
architecture  using  standards  which  support 
user  requirements 

*  Standard  protocols  for  networking  to 
ensure  interoperability  throughout  DoD 


TECHMCAL  WORKINO  GROUP  #2 
ARCHITECTURE  NEEDS 


•  Need  ADA  to  bo  Improved  to  support 
M&S  requirements 

•  Until  ADA  is  Improved,  the  community 
needs  language  flexibility  to  support 
peraileiism,  graphics  user  interfaces, 
better  I/O,  multi  tasking,  affordable 
compilers  (etc) 

•  Develop  opeivsystem  architecture 
standards  to  support  M&S 
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TECHMCAL  WORKING  GROUP  «2 

ARCHITECTURE  NEEDS  (Continued) 


*  Develop  standard  protocols  for 
networking  to  ensure  interoperability 
throughout  DoD 

*  Develop  standard  Interfaces  for  selected 
classes  of  model  subsystems  (used 
throughout  the  T&E  process  to  promote 
software  reusability  and  agency 
collaboration 
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TECHNICAL  WORKINO  GROUP  «2 

ARCHITECTURE  NEEDS  (Continued) 


Develop  standard  models  for  selected 
classes  of  functions  (e.g.,  environment: 
terrain,  multi-spectral  clutter,  weather, 
EM  propagation,obseurants,  human 
behavior)  used  throughout  the  T&E 
process  to  promote  software  reusability, 
agency  collaboration,  and  eliminate 
duplication  of  effort 
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TECHMCAL  WORKING  GROUP  «2 

INFRASTRUCTURE  NEEDS 


•  Fix  responsibility  for  T&E  M&S  technology 
development  on  specific  DoD  Centers  of 
Exceiience 

•  Designate  dedicated  service/agency  focai 
points  for  M&S 

•  Establish  DoD  baseline  training  levels  for 
M&S  professionals 

•  Make  SEMike  training  affordable  and 
available  to  community 
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TECHMCAL  WOraONQ  GROUP  « 
INFRASTRUCTURE  NEEDS  (ConSnued) 


•  EttabUshDoD/StrvlOGelMringhousGSfor 
MAS  and  databRM  Inroniwtion 

•  DGVGtopanMilomRtGdmMntldrthG 
mGtnbtrt  of  ttiG  MAS  community  to 
communicolo  and  rotrlava  tha  toitormation 
from  daarlng  houaaa 

•  Eatabllah  DoD  baaalina  training  lavala  for 
MAS  Drofoaalonala 

-  Mam  SEHIka  training  affordabla  and 
avallabla  to  community 


TECNMCAL  WORKING  GROUP  #2 

INFRASTRUCTURE  NEEDS  (Concluded) 


•  Develop  an  affordable  high  apaed, 
multl-leval  aecure  network  connecting  TAE 
facilities. 
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TECHMCAL  WORKINQ  GROUP  i2 
INTEGRATED  INFRASTRUCTURE  VISION 


Capitalize  on  OoD  davalopmant  of  Canters  of 
Excellence  for  M&S  technology 

Centralized  dedicated  focal  points  for  M&S  for  each 
aervice/agency 

Personnel  training  for  managers,  analysts,  engineers 
to  estabitsh  a  corps  of  M&S  professionals 

Accessible  means  to  identify  and  acquire 
Information  about  M&S  and  databases 

Affordable  high  speed  multi-level  secure  network 
connecting  T&E  facilities 
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INTEGRATED  VISION 
AND  NEEDS 

TECHNICAL  WORKING  GROUP  3 


TECHMCAL  WORKINO  GROUP  #3 


INTEGRATED  VISIONS  AND  NEEDS 


TECHNICAL  WORKING  GROUP  «3 

INFRASTRUCTURE 


•  Free  on>line  databases  Of : 

•  models  and  simulators 

-  terrain  and  environmental  data 
These  would  Include: 

-  Indexes 

-  descriptions 

•  access  procedures 


TECHMCAL  WOmONO  OROI»  #3 
INFRASTRUCTURE  (Cont'd) 


•  Opdn  BB/«xehang*  media.  Including 
-models 

-  slmutstions 

-  environmental  data 
-terrain 

-  aignatures 

-  vulnerabilitiea 

-  performance  characteristics 

•  On-line  heip 

•  Modei/Simuiator  user  groups 

•  SImpie  giobai  network  controi  center(s) 
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TECHNICAL  WORKING  GROUP  «3 

iNFRASTRUCTURE  (Cont'd) 


*  Reconfigurabie  network  with  traceabiiity 

•  Tracking  system  for  capturing  and 
certifying  system  test  configurations 

*  Open  multi-vendor  distributed  processing 
support 

•  Self  documenting  code 
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TECHMC AL  WORKMQ  GROUP  #3 

INFRASTRUCTURE  (Conc'd) 


*  Better  characterization  of 

-  command  &  control 

-  environments 

-  intelligence  fusion 

-  etectromagnetic/iicoustic  warfare 
•  human  decision  making 

•  Relation  of  the  above  to  combat 
effectiveness 


TECHNICAL  WORKING  GROUP  #3 

ARCH,  STANDARDS  AND  PROTOCOLS 


Databases  with  standardized 

-  data  dictionary  and  nomenclature 

-  sources  and  confidence  levels 

•  access  levels 
-  security 

•  need'to-know 

Databases  and  models  with  improved 

•  distributability 

•  reuse 

•  interoperability 

•  reconfigurability 

-  accessibility 


TECHMCAL  WORKINQ  QROUP  #3 
ARCH,  STANDARDS  AND  PROTOCOLS  (Cont'd) 

*  Supports  diffarant  execution  modes 

•  real-time 
-  non  real-time 

*  Standardized  model  analysis  and  architecture 
design  methodology 

*  Easy  data  storage 

*  Multi-level  network  security  and  control 

*  Automated  scenario  generation 


TECHNICAL  WORKING  GROUP  »3 

ARCH,  STANDARDS  AND  PROTOCOLS  (Cont'd) 

*  A  more  efficient  model  V&V  process  through: 

•  automation  (e.g.,  CASE) 

-  sharing  validation  data 

-  configuration  management/management 
audit 

*  Standardized  communications  protocols 
including: 

-  multi-rate 

-  multi-cast 

-  varying  bandwidth  (e.g.,  underwater) 


TECHMCAL  WORKmG  GROUP  #3 
ARCH,  PROTOCOLS  AND  STANDARDS  (Conc'd) 


•  Open  20/3D  visualization  standards  with 
reusable  military  graphic  building  blocks 

•  Easy  data  storage 

•  Visual  programming/code  generation,  with 
automated  consistency  checking 

•  New  visualization  applications  for: 

-  briefing/rapid  debriefing 

-  systems  operations/controi 
•  data  extraction  and  analysis 
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TECHNICAL  WORKING  GROUP  «3 

CAPABILITIES 


•  Open  multi-vendor  distributed  processing 

•  Open  visualization  standards  and  building 
blocks 

•  Visual  programming/code  gen  with 
consistency 

•  Self  documenting  code 

•  Multi-ievei  network  security  and  control 

•  Reconfigurable  network  with  traceability 

•  Standard  comm  protocois 
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TECHMCAL  WORKMQ  QROUPiS 

CAPABILITIES  (Conc'd) 


•  B«tt«r  models:  environment,  Intel,  C2, 
human,  EW-AW 

•  Better  accommodation  of  varying  resolution 

•  More  efficient  nwdel  V&V 

•  Test  configuration  tracking  and  certifying 

•  New  visualization  applications 

•  Automated  scenario  generator 

K-n 


APPENDIX  L 
POLICY  NEEDS 

POLICY  WORKING  GROUP 


TAE  HAS  REQUmaiBITS  WORKSHOP 


POLICY  NEEDS 
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TAE  MAS  REOUIREHENTS  WORKSHOP 

PRIORITlZATtON  METHOD 


Policy 

Mgmt 

Ttch 


BANOI 


BAND2  I 
I  ^ND3 


Today  Tomorrow  Futura 


TtE  MAS  REQUmaiENTS  WORKSHOP 

PRIORITY  BANDS 


•  BAND1:  Things  that  should  be  done  and 
funded  now 

•  BAND  2:  Things  that  are  important  in  the 
near  term  but  can  be  done  and  funded  in 
phases 

•  BAND  3:  Things  that  are  needed  in  the 
future  but  may  need  further  development 


TAE  MAS  REQUIREMENTS  WORKSHOP 

POLICY:  BAND1 


*  Establish  central  M&S  authority  to  which  all  T&E 
M&S  activities  must  report 

*  Develop  a  Master  Plan  to  use  M&S  for  T&E 

*  M&S  should  be  used  to  support  the  T&E  process 
throughout  the  system  life  cycle 

•  V&V  design  requirements 

•  development  to  deployment 
-  operational  assessments 


TAE  HAS  REQUREMENTS  WORKSHOP 

POLICY:  BAND  1  (concluded) 


•  DoD  components  shall  establish  V,  V&A 
policy,  procedures,  and  responsibilities 

•  Training:  A  M&S  career  track  will  be 
established,  to  Include  M&S  course  work 

•  Share  models  and  data  across  DoD 

•  Standardized  data  definitions 

•  Standard  database  elements 


TAE  MAS  REQUIREMENTS  WORKSHOP 

POLICY:  BAND  2 


All  M&S  used  In  support  of  T&E  will  be 
accredited  by  the  user 

M&S  should  be  developed  using  a  common 
open  systems  architecture  using  associated 
standards  and  protocols 

Architecture,  standards,  and  protocols  must 
provide  for  variable  fidelity  (as  dictated  by 
user  needs) 


TAE  MAS  RECMIIREMENTS  WORKSHOP 

POLICY:  BAND2(conrd) 


*  M&S  Software  should  meet  established 
standards  and  metrics  (i.e.,  MIL*STD*2167A, 
Army  Software  Test  and  Evaluation  Panel 
process,...) 

*  M&S  users/developers  shall  use  valid  data 

*  M&S  developers  will  coordinate  data 
requirements  with  DoD  Offices  of  Primary 
Responsibility  prior  to  expenditure  of  funds 


T&E  M&S  REOUIREMENTS  WORKSHOP 

POLICY:  BAND  2  (cont'd) 


*  Require  the  use  of  standard  databases  (i.e.,  Intel, 
terrain,  atmosphere,  climate,...) 

*  Producers  of  standard  databases  will  comply  with 
standard  database  elements  definitions 

*  Establish  "library"  of  validated  M&S  modules  (at 
OoD  or  Service  levels) 

*  Establish  central  M&S  "Lessons-Learned"  Library 

*  Encourage  distributed,  real-time  processing  with 
Man-in-the-loop/HW-in-the-loop  capability 
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T*E  MAS  REQUmEMBITS  WORKSHOP 

POLICY:  BAND  2  (concluded) 


*  M&S  should  be  used  when  live  tests  ere 
limited  or  cannot  be  conducted 

*  The  contractor  should  provide 
non-proprietary  deliverable  software 
packages 


TAE  MAS  REQUiREMEMTS  WORKSHOP 

POLICY:  BANDS 


•  M&S  will  be  designed/modified  with  the 
following  in  mind;  portability,  interoper¬ 
ability,  credibility,  usabiiity,  flexibility, 
reusability,  internetting  and  security,  and 
linking. 

•  All  M&S  architectures  should  have  the 
capability  to  Import  and  export  standard 
databases  produced  by  DoD  Offices  of 
Primary  Responsibility 

•  M&S  will  consist  of  a  standardized 
documented  version  which  will  remain  intact; 
changes  to  the  standard  will  occur  to  other 
versions 
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TftE  MAS  REQUmEMENTS  WORKSHOP 

POLICY:  BAND3(eonrd) 


Intemot  should  bs  capable  of  providing  data 
links  for  sub-surfaca/surfaca/air/apaca 

M&S  will  allow  vertical  and  horizontal 
integration  (a.g..  Aggregate  Laval  Simulation 
Protocol) 

*  Capable  of  playing  roles  in  Joint-service 
scenarios  providing  essential  degree  of 
realism 

•  Perform  real-time  data  collection,  reduction, 
visualization,  and  distribution 
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T«E'  MliS  REQUIREMENTS  WORKSHOP 
POLICY:  BAND  3  (concluded) 


Establish  high  volume  networks  for  transfer  of 
multi-level  classification  test  data 


M&S  shall  relate  test  results  to  combat 
effectiveness  and  battle  outcomes 
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MANAGEMENT  NEEDS 

MANAGEMENT  WORKING  GROUP 


TIE  MftS  REQUREMBITS  WORKSHOP 


MANAGEMENT  NEEDS 


TEE  MSS  REOUmEMENTS  WORKSHOP 

MANAGEMENT:  BAND1 


•  Promots  oarly  Involvement  by  management, 
user,  developer 

•  Provide  resources 

•  Establish  service  proponency 

•  Standardise: 

-  architecture 

-  databases 

•  definitions 

•  documentation 

•  Standardize  W&A  Process 
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TftE  M&8  REQUmEUBITS  WORKSHOP 

MANAGEMENT:  BAND  2 

Provide  relief  or  waivers  from  ADPE  and 
communications  network  equipment 
procurement  regulations 

Provide  centralised  coordination  and  allow  for 
decentralized  execution 

Expand  to  all  phases  of  T&E 

Educate 

•  publicize 

•  train 

Develop  and  use  multi-service  expertise 
Distribute  products 


TSE  MSS  REQUIREMENTS  WORKSHOP 

MANAGEMENT:  BANDS 

Develop  &  maintain  modeis/databases 
Improve  network 

Establish  multi-level  security  capability 

Recognize  and  coordinate  new  technology 
insertion 


.  m  IMS  REQUmEMBITS  WORKSHOP 

'  POLICY  &  MANAGEMENT  CONCLUSION 


From  a  Policy  and  Managamant  parspactive, 
Modaling  and  Simulation  in  support  of  T&E 
should: 

-  Raduca  costs 

-  Raduca  risks 

-  Incraasa  afficiancy 
•  Raduca  schadulas 

-  Improve  T&E  process  credibility 
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APPENDIX  N 


TECHNICAL  NEEDS 

TECHNICAL  WORKING  GROUP 


TAE  MAS  REQUIREIIENT8  WORKSHOP 


TECHNICAL  NEEDS 


N-t 

TAE  MAS  REQUIREMENTS  WORKSHOP 
M&S  AND  DATA  MANAGEMENT  NEEDS 

Band  1 

*  Establish  clearing  houses/repositories  (free 
and  easy  access,  bulletin  boards) 

*  Develop  &  disseminate  M&S  standards 

*  Establish  M&S  user  group 
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T&E  MftS  REQUREMEHTS  WORKSHOP 

M&S  PROCESS  NEEDS 


Bandl 

*  Place  responsibiiity  for  T&E  M&S 
technology  development  on  specific  DoD 
Centers  of  Excellence 

*  Designate  dedicated  service/agency  focal 
points  for  M&S 

-  centralized  management 

•  decentralized  execution 

*  involve  warfighter  and  T&E  M&S 
communities  early  In  acquisition  processes 
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TSE  M&S  REQUIREMENTS  WORKSHOP 

M&S  W&A  NEEDS 


*  Establish  W&A  process  that  provides: 

•  standardized  documentation 

-  formal  Configuration  Management 

-  shared  information 


TftE  y&S  REQUmEIIEIITS  WORKSHOP 
M&S  W&A  NEEDS 


Band  2  •  Devalop  automatad  V&V  procassas  and 
support  toola 

•  Davalop  a  W&A  procass  for  tha  futura 
that  strasaas  both  stand-alona  and 
intaroparabla  diatributad  aimulations 


T4E  M4S  REQUIREMENTS  WORKSHOP 

PROFESSIONAL  DEVELOPMENT  NEEDS 


Band  2 

•  Establish  DoD  basalins  training  levels  for 
M&S  professionals 

•  Establish  grade  structuras/skill 
raquiramanta/caraar  paths 

•  Establish  appropriate  training  programs  for 
M&S  professionals  and  practitioners  within 
the  T&E  community 

•  Make  training  affordable  and  availabla  to 
community  (a.g.,  Software  Engineering 
Institute) 


TAE  MAS  REQUIREMENTS  WORKSHOP 

MODEL  AND  SIMULATION  NEEDS 


Bafid2 

Develop  standerd  models  for  selected 
classes  of  functions  used  throughout  the 
T&E  process  to  promote  software 
reusability,  agency  collaboration,  and 
eliminate  duplication  of  effort 

•  terrain 

•  multi-spectrai  clutter 
-  weather 

•  EM  propagation 

•  human  behavior 

Relate  system  measures  of  performance 
to  combat  effectiveness/battle  outcome 
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TAE  MAS  REQUIREMENTS  WORKSHOP 

MODEL  AND  SIMULATION 

_ NEEDS  (continued) _ 

Band  2 

*  Encourage  M&S  development  with  the 
following  attributes: 

•  very  high  fidelity 

-  varying  resolution/scale/scope 

-  interoperability  (between  models) 

-  robust  capability 

-  muiti-spectrai 

-  soft  kill/hard  kill 

-  ease  of  use 

•  standardized  portability 


T&E  y&S  REQUREHENTS  WORKSHOP 

MODEL  AND  SIMULATION 
_ NEEDS  (Concluded) 

Band  2 

-  object*ori0nted  and  open  systems 
architecture 

•  real-time  operations 

-  man-in-the-loop 

-  hardware-in-the-loop 

•  distributablllty 

-  reuse 

-  reconfigurability 

-  accessibility 
Where  It  makes  sense 


T&E  MSS  REQUIREMENTS  WORKSHOP 

M&S  INTERFACE  NEEDS 


Band  2 

*  Establish  data  interface  standards  to 
include: 

•  data  definition 

-  format 

-  joint  data/object  dictionary 

•  Establish  data  interface  standards  with 
capability  for: 

•  on-line  test  cases 

-  network  and  stand-alone 
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T«E  M&S  REQUIREMENTS  WORKSHOP 

M&S  INTERFACE  NEEDS  (Concluded) 


Band  2 

Develop  standard  interfaces  for  selected 
classes  of  model  subsystems  (used 
throughout  the  T&E  process)  to  promote 
software  reusability  and  agency 
collaboration 


T&E  M&S  REQUIREMENTS  WORKSHOP 

M&S  DATABASE  NEEDS 


Band  2 

Develop  improved  capability  for  M&S 
databases  to  include: 

•  standardized  data  definitions  and  formats 

•  common  data  on>line  dictionary 

•  intelligent  access  procedures 

•  validation  audit  trail 

>  Configuration  Management,  maintenance 
and  distribution 

-  multMevel  security 

•  accessibility 

-  multi'levei  fidelity 


TftE  H&S  REQUIREMENTS  WORKSHOP 


M&S  DATABASE  NEEDS  (Concluded) 


BAND  2 

•  Establish  common/shared  databases  for 
common  threat,  environment,  terrain, 
validation  test  data,  etc. 


T&E  M&S  REQUIREMENTS  WORKSHOP 

M&S  NETWORK  NEEDS 


Band  2 

*  Develop  an  affordable  high  speed, 
multi-level  secure  network  connecting 
T&E  facilities  from  existing  networks 

*  On-line  knowledge-based  asset  manager 

*  Reconfigurable  network  with  traceability 
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T&E  MAS  REQUIREMENTS  WORKSHOP 

M&S  NETWORK  NEEDS  (Concludsd) 


Band  2 


*  Establish  improved  network  capabilities 
in  the  areas  of: 

•  net  management 

-  asset  readiness  check 

-  standardized  cost  procedures 

-  tunable  data  rates  f 

-  minimum  latency 

-  embedded  muitMevel  security 

-  on*iine  context-sensitive  help 
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T&E  M&S  REQUIREMENTS  WORKSHOP 
M&S  TECHNOLOGY  NEEDS 
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Band  2 

*  Focus  M&S  technology  investments  in 

-  visual  programming 

-  visualization/imaging  virtual  reality 

•  data  compression 

•  dynamic  user-defined  environment 
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TftEMAS  REQUIRBIBITS  WORKSHOP 

M&S  SOFTWARE  DEVELOPMENT  NEEDS 


Bands 

Support  standard  SW  davalopment 
mathodology  with  SW  support  anvironmant 
including  automatad  documantation 

Support  distributad  axacution  across 
networks  or  massiveiy-paraliai  systems 


T4E  M&S  REOUIREMENTS  WORKSHOP 
M&S  MULTILEVEL  SECURITY  NEEDS 


Band  3 

•  M&S  community  needs  access  to  on-going 
efforts  to  develop  muiti-ievel  security 
capability  to  ascertain  if  these  efforts  will 
meet  sp^ific  needs  of  the  M&S  community 

-  Networks 

-  Operating  systems 

•  Database  Management  Systems 

-  High  speed  encryption/decryption 


M 


T&E  M4S  REOUHffiiBfTS  WORKSHOP 

M&S  LANGUAGE  NEEDS 


Band  3  •  Need  Ada  to  be  Improved  to  support 
M&S  requirements 
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T4E  MSS  REQUIREMENTS  WORKSHOP 

M&S  LANGUAGE  NEEDS 


*  Until  Ada  is  improved,  the  community 
needs  language  flexibility  to  support 
parallelism,  graphics  user  interfaces, 
better  I/O,  multi-tasking,  affordable 
compilers,  etc. 
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OVERVIEW  OF 
REQUIREMENTS 
SURVEY  FOR 
TEST  AND  EVALUATION 
MODELING  AND 
SIMULATION 


Overview  of  Requirements 
Survey  for  Test  and 
Evaiuation  (T&E)  Modeiing 
and  Simulation  (M&S) 


Background 


On  21  June  1991  Deputy  Secretary  of  Defense  signed  a 
memorandum  which  directed  that  DoD  make  more 
effective  use  of  models  and  almulatione.  This  memo 
assigned  rssponstoility  to  Under  Secretary  of  Defense 
(Acquisition),  with  advice  from  the  Executive  Council  for 
Modeling  and  Simulation  (EXCIMS). 

One  goal  of  fhe  Defense  Modeling  and  Simulation  (M&S) 
Initiative  is  to  Invest  In  joint  hlgh«retum  M&S  activities. 

EXCIMS  investment  guidance  includes  the  following 
objectives: 

•  Promulgate  Standards  to  Promote  interoperability  of 
the  components  of  the  M&S  Environment 

•  Support  development  of  databases,  tools  and 
methodologies  for  community>wide  use 


Background  (Cont'd) 


•  Promot«0«v«lopnMm  of  a  Communications 
infrastnictura  to  Support  imagration  of  Joint  Modaling 
and  Simuiation  Activhlaa 

•  Facilitata  Communtty>Wlda  Coordination  and 
Information  Sharing 

•  MSS  naads  and  capability  gaps  wars  not  wail  dafinad 
for  usa  In  JudgingM&S  proposals  for  FY  91/92  DMSO 
funding 

•  M&S  raquiramarns  survay  gatharad  information 
naadad  to  furthar  dafina  naeds/capability  gaps 
directly  from  T&E  community 
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Survey  Goals 


Gather  information  on  TSE  M&S  naads  from  M&S 
davaiopars  and  users  to  usa  as  input  to  the  T&E  M&S 
Workshop. 

Results  of  survey  will  help  answer: 

Which  M&S  invastrnams  wili  most  benefit  T&E 
community  (short  term  and  long  term)? 

How  can  invastrnams  in  M&S  activltias  help  solve 
T&E  problems  and  reduce  net  costs  of  T&E 

What  policy/managamant  Issues  should  be 
addressed  to  improve  affactivanass  of  M&S  used  for 
T&E? 


Specific  Objectives  of  M&S  Survey 


•  OtltniiiM  currant  and  lona-mig«probl«ns/iiinltations/ 
capabiNtysiaps  (cxpraaaaa  by  imiHipla  aaivicat)  bi  existing 
models  ma  simulations. 

•  Determine  how  tee  can  explod  sdvances  In  technologies  (e.g. 
Networks)  to  "improve"  IMS  for  TSE  in  multiple  services. 

"Improve"  IMS  for  TSE  means  reduce  net  cost,  Increase 
saf^,  hnprove  aecurfty,  reduce  environmental  impact 
and/or  tm^ve  thoroughness  of  T&E  through  IMS. 

•  Determine  other  IMS-relsted  activities  that  would  "improve” 
application  of  IMS  for  TSE  m  multiple  services. 


•  Provide  a  Itasis  with  which  to  enter  the  T&E  Requiremems 
Workshop. 
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Survey  Execution 


•  Prepared  Introductory  briefing  and  designed  a 
qurmionnaira 

•  Sent  questionnaires  to  eleven  TSE  sites  and  emered 
Information  from  completed  questlonnairas  imo 

database 

•  Held  orvalte  meetings  with  questlonnalrarawondents 
(focus  groups)  to  diaiogue  and  expand  on  information  in 
questionnaires 

•  Analyxed  and  aummartxed  intormatlon  gathered  during 
survey 


T&E  Survey  Sites 


•  NAWC  AO.  PAX  RIvm  (Navy) 

•  TECOM  and  TRAC,  Whita  Sanda  Miaalia  Rang#  (Army) 

•  NAWC  WO.  ChbM  Laka  (Navy) 

•  NAWC  WO.  Point  Mugu  (Navy) 

-  Aaroapaoa  Corp.  and  Edwarda  APB.  Loa  Angalaa  (Air  Forca) 

•  AFOTEC,  Kirkland  Air  Forca  Baaa 

•  TEXCOM/OPTEC.  Fort  Hood  (Army) 

•  USAEPQ,  Fort  Huachuea  (Army) 

•  IISIC,  Radatona  Araanal  (OiA) 

•  Eglin  AFB  (vidao  talaconfaranca 
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Analysis  of  Survey  Information 


a  Analyzad  data  from  quaationnairaa  to  MantHy  muNisarvica 
iaauaa  and  datarmina  how  aida-apraad  thaaa  iaauaa  ara 

•  IdantHiad  Itay  phraaaa"  (rafiacting  managamant  Iaauaa, 
policy  iaauaa  and  tachnical  iaaua^  from  aach  quastionnaira 
and  from  focua  group  moating  notos 

•  Countad  numbar  of  occurroncaa  of  aach  kay  phraaa-if  a  kay 
phraaa  (or  aquivllant  w<^a  rafiacting  tho  aama  iaaua) 
appaarad  ona  or  mora  timaa  in  a  quaationnaira  or  waa 
mantionad  ona  or  mora  timaa  during  a  focua  group  maating, 
that  waa  countad  aa  ona  occurronca  for  that  kay  phraaa 

a  Countad  oocurranoaa  of  aach  laval  of  raniking  for  tha  Hama 

(auggaaatad  MAS  naada  and  Iaauaa)  liatad  In  priorltazation 
tablaa  of  quaationnakaa 


Summarizing  Survey  Resuits 


•  SummarliMl  toy  pliiMM  (issuM) 

-  Grauptd  toy  phrMM  into  managwiwnt,  poliqr 
ondtoehnicaiiMUM 

•  LModtodigroupof  toyphroMtlnordarof 
dacraMing  numtert  of  occtmoncm  (toy 
phroMs  Whhmof  occunonc— atoodof  toy 
phraaaa  with  fa«Mr  oeeunancaa) 

•  AtoyphfaaahadtooeeurhKiiiaationnaira 
and/or  foeua  group  diaeuaaiona  from  two  or  mofo 
aarvicaa/agancias  to  t)a  indudad  in  aummary 
Information 

-  RMlaeadaach  toy  phraaa  with  thaiaaua  that  it 
rafiaeta  and  gavaan  axpianation  of  tha  iaaua 
whan  appropriata 

•  Faetorad  priorttiaatlon  (ranking)  data  from  aurvay 
quaationnairaa  into  aummary  intormation 


Summarizing  Survey  Resuits  (Cont'd) 


Policy 


Managamant  Tachnical 


•  Tha  aummarlaa  of  aurvay  raaulta  hava  baan  providad  to 
chairman  of  tha  working  groupa 

•  iaauaa  will  ha  axpandad  upon  and  priorftizad  during 
working  group  diaeuaaiona 


